Synthesis of Fault Tolerant Neural Networks

This paper evaluates different strategies for enhancing (partial) fault tolerance (PFT) of feedforward artificial neural networks (ANNs). We evaluate a continuum of strategies between the two extremes (i) Replicating a minimal seed network to achieve a final desired size with the resultant PFT and (ii) Starting with the desired (larger) size but using modified training algorithms to achieve higher PFT (without any replications) The idea is not to replicate the minimal network but somewhat larger-than-minimal network and evaluate the effect on the PFT of the resulting network. In other words we investigate the optimal size of the seed network (which gets replicated) that achieves the highest PFT for a fixed final size (i.e., the total number of units and connections). The data demonstrate that replicating larger-than minimal networks yields higher PFT for the same final size (as compared with either replicating the minimal network or starting off with the final target size and employing modified training but not using replications at all). Furthermore, it is seen that when the size of the seed network exceeds some threshold, the PFT for a given final size typically worsens. Thus, there is an optimal size for the seed network. We provide qualitative explanation of this and allied phenomena.